Skip to content

the introspection cache and buildkey solution#1304

Open
Zetazzz wants to merge 29 commits into
mainfrom
perf/multi-tenancy-cache-v5
Open

the introspection cache and buildkey solution#1304
Zetazzz wants to merge 29 commits into
mainfrom
perf/multi-tenancy-cache-v5

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

This PR implements a new multi-tenancy optimization strategy without changing Graphile/Crystal internals, centered on exact-match buildKey-based handler reuse. Build keys are derived from connection identity, schemas, and role inputs, allowing requests with identical build inputs to share the same Graphile handler while preserving strict isolation semantics. The PR also adds perf stress-test coverage, including shape-variant perf tooling, so the new caching path can be exercised and evaluated under load without reintroducing template sharing or SQL rewriting.

if (typeof opts.connectionString === 'string') {
try {
const url = new URL(opts.connectionString);
const host = url.hostname || 'localhost';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these types of defaults should be done the way we do other pgpmjs/types or pgpmjs/env — and I think we have the same for our graphql server, env/types

https://github.com/constructive-io/constructive/tree/main/graphql/env
https://github.com/constructive-io/constructive/tree/main/graphql/types

try {
const url = new URL(opts.connectionString);
const host = url.hostname || 'localhost';
const port = url.port || '5432';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

* 1. connectionString-based (production — via pg-cache's getPgPool)
* 2. individual fields (fallback for pools created with explicit fields)
*/
export function getPoolIdentity(pool: Pool): string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also feels like a utility, would we use this anywhere else?

@socket-security

socket-security Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgraphile-connection-filter@​1.14.17810010096100

View full report

@yyyyaaa

yyyyaaa commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

We just pushed the full multi-tenant capacity program to #1325 (measured on a production-shaped rig: 48 tenant DBs, 61k pg_class rows in one database), and since both PRs attack the same problem through the same seam, posting a comparison here so we can coordinate before either side invests more.

First — this PR's skeleton is the right shape, and #1325 independently converged on the same one: normalized build-input key + single-flight creation + svc_key→key mappings with epoch/per-database flush. The disposal WeakSet double-release guard matches what we ended up with too.

Three measured differences that changed our conclusion:

1. Sharing scope. computeBuildKey keys on physical schema names (order-preserved), so two tenants never share a handler — this dedupes aliases of the same tenant-API, not tenants. At the 61k-row catalog each handler still retains ~1.3GB (≈21.9KB per pg_class row), so a 2GB node holds one resident handler either way. #1325 keys on schema shape (canonical-tenant build + identifier rewrite at the pool seam, no search_path, SDL byte-identical by MD5): same-shape tenants collapse to one handler. Measured end-state: 47 tenants on one 2048MB node — 104,818/104,818 OK over 30 min of authenticated zipf @50rps, p99 21ms, heap peak 195.7MB (9.6%), zero cross-tenant bleed.

2. Introspection cost. Unless we misread the diff, there's no introspection caching in this PR (the "introspect" hits are the buildKey doc comment and the perf tooling's schema discovery) — every handler build still runs full-catalog introspection and retains the whole graph. That cost turned out to be the wall, and it's fixable independently of the caching strategy: #1325 rewrites the introspection SQL pre-parse to only the namespaces the instance serves → 1,305.6MB → 14.7MB retained per instance (87×), cold build 7.2s → 0.42s, PG-side spike +1,283MiB → +17MiB. Notably, createIntrospectionFilterPool wraps dedicated instances too — if the no-SQL-rewriting stance here is a hard requirement for some deployments, adopting just the filter would take this PR's per-handler cost to ~15MB without touching its caching model.

3. Cache bounds under ~GB entries. max=50 + 1-year TTL + dispose without an in-flight drain is the configuration the program measured as the OOM path at this catalog size (50 × 1.3GB potential residency; and dispose racing live requests). One concrete bug we both hit: httpServer?.listening is always false for a createServer()'d instance that never .listen()s, so the close() branch never runs. #1325 replaced this layer with a heap-aware capacity law (R = clamp(min(⌊(H−B)/I⌋, ⌊(H−B−T)/I⌋+1), 1, 50)), evict-before-build + bounded drain with request refcounts, and a memory governor that degrades to 503 + Retry-After instead of aborting — each piece traceable to a specific measured crash in scripts/scale-validate/VALIDATION-REPORT.md.

Practical notes:

Full methodology and raw numbers: SIZING.md + VALIDATION-REPORT.md on #1325. Happy to walk through any of it, or to help wire the introspection filter into this branch if we want an apples-to-apples rerun of its perf suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants